Journal: bioRxiv

Article Title: Web engine for tumor pathology image retrievals on massive scales

doi: 10.1101/2025.10.25.684566

Figure Lengend Snippet: The framework comprises two modules: Development and Retrieval. a, The engine development has two stages: model training and index generation. Stage 1 trains a prediction model composed of an image encoder, attention map, and decoder using TCGA data from 32 tumor types. The input to the image encoder is H&E whole-slide images, and the decoder output includes a binary indication of gene mutation functions in cancer progression and average expression of pathway gene sets. Stage 2 indexes patch vectors encoded from the whole slide images from the NCI Lab of Pathology, TCGA, CPTAC, and spatial transcriptomics studies. b, Two retrieval applications. For a query H&E image patch or region, the Image-to-Image retrieval returns similar image patches and associated whole slides from the indexed database. If users select images with spatial transcriptomics (ST) data, HERE returns images that closely match the query and the corresponding gene expression profiles from ST detection spots. The Transcriptomics-to-Image retrieval finds H&E slides with paired ST data where the query gene has high expression levels in ST detection spots associated with specific image feature clusters from those slides.

Article Snippet: Spatial transcriptomics (ST) data paired with H&E images also makes it possible to implement a Transcriptomics-to-Image module, which returns image features associated with the high expression level of a query gene name (e.g., SERPING1 ) ( ).

Techniques: Mutagenesis, Expressing, Gene Expression

Journal: bioRxiv

Article Title: Web engine for tumor pathology image retrievals on massive scales

doi: 10.1101/2025.10.25.684566

Figure Lengend Snippet: a , An example query image from a tumor with hypoxia indicated by anti-Pimonidazole staining . The right side shows examples of matched image patches (green squares) within an H&E slide with spatial transcriptomics (ST) data. b , Transcriptomics heatmap of ST detection spots from the top three ST profiles, each representing a distinct cancer type. Expression values are variance-stabilizing transformed values relative to levels in all ST spots and sequencing depth per slide . Only genes with expression values greater than 5 in at least three spots across all match patches are shown, and genes are ranked by mean values across all spots. Within each cancer type, columns (spots) are organized by hierarchical clustering with correlation distances. Only spots with expression values greater than 5 in at least three genes are shown. c , Glycolysis gene set enrichment. Along the x-axis, all genes are ranked from high to low by mean expression value (lower y-axis) among all ST detection spots returned by the image query. Members of the glycolysis hallmark gene set are indicated by horizontal blue lines in the middle of the plot. The top y-axis plots the glycolysis enrichment score at each gene rank. The p-value was computed through the one-sided permutation test with 1000 randomizations. d , Gene sets with higher enrichment scores than hypoxia.

Article Snippet: Spatial transcriptomics (ST) data paired with H&E images also makes it possible to implement a Transcriptomics-to-Image module, which returns image features associated with the high expression level of a query gene name (e.g., SERPING1 ) ( ).

Techniques: Staining, Expressing, Transformation Assay, Sequencing

Journal: bioRxiv

Article Title: Web engine for tumor pathology image retrievals on massive scales

doi: 10.1101/2025.10.25.684566

Figure Lengend Snippet: a , Screenshot of the 3D UMAP of patch encoding vectors from H&E images paired with spatial transcriptomics. The full interactive UMAP is available at https://hereapp.ccr.cancer.gov/ST_CONCH_umap3d.html . Some image patch clusters or local regions comprise image patches from mostly one or two ST profiles (circle highlights), typical of batch effects. b , Statistical links between gene expression and image feature clusters. For each ST cluster, we counted the number of genes with FDR < 0.05 (Benjamini-Hochberg corrected from the two-sided Wilcoxon rank sum test). The histogram of gene count values across image clusters from all ST profiles is shown. The total number of clusters evaluated was 1,039, and the total number of genes was 11,137. c , Gene set enrichment analysis. For Cohen’s d profile for each ST cluster, we performed gene set enrichment analysis. The X-axis presents the fraction of ST clusters above which a GO_BP term is enriched (GSEA q-value < 0.05). The left Y-axis presents the fraction of GO_BP terms enriched above the threshold on the X-axis for real and randomly permuted data. The right Y-axis presents the False Discovery Rate, computed as the (Random GO_BP term fraction) / (Real GO_BP term fraction). d , The H&E image from the human lung tumor region shown in , which has high expression of C1R , C1S , and SERPING1 . e , In-vitro growth of B16F10 cancer cells in culture, measured by XTT assay. Dots and error bars represent mean and standard deviations (n = 3 cell culture replicates). Metabolic activity is measured as optical density at 492 nm (read) divided by the value at 620 nm (reference)

Article Snippet: Spatial transcriptomics (ST) data paired with H&E images also makes it possible to implement a Transcriptomics-to-Image module, which returns image features associated with the high expression level of a query gene name (e.g., SERPING1 ) ( ).

Techniques: Gene Expression, Expressing, In Vitro, XTT Assay, Cell Culture, Activity Assay

Journal: bioRxiv

Article Title: Web engine for tumor pathology image retrievals on massive scales

doi: 10.1101/2025.10.25.684566

Figure Lengend Snippet: a , Associations between gene expression and image features. Hierarchical clustering was applied to Spatial transcriptomics (ST) data from a human lung tumor to organize image patches around ST detection spots into eight clusters (left panel; each cluster is a different color). For a given query gene (e.g., C1R and SERPING1, center panel) and each image cluster (for example, cluster #2 (blue) in left panel), the expression difference among ST detection spots within the image cluster region and spots outside the cluster region is quantified using the Cohen’s d value (right panel). Testing each of all possible query genes against each of every ST profile cluster will generate the result matrix (bottom panel). b , Complement Activation gene set enrichment. Along the x-axis, all genes are ranked from high to low by Cohen’s d values (bottom Y-axis) computed for cluster #2 of the ST profile in panel a. Members of the “complement activation” pathway from Gene Ontology biological processes (GO_BP) are indicated by horizontal blue lines in the middle of the plot. The top y-axis plots “complement activation” enrichment scores at each gene rank. The P -value is computed through the one-sided permutation test (1000 randomizations). c , Top 20 GO_BP terms associated with image features. For each term, the Y-axis presents the fraction of ST profile clusters whose Cohen’s d gene scores are significantly enriched (False Discovery Rate < 0.05). Multiple GO_BP terms related to similar biological processes are grouped with the y-axis presenting mean values across all merged terms. The star marks the complement activation pathway discussed in the main text. d , Cohen’s d heatmap of complement activation genes. Columns are labeled with each ST profile’s cancer type and the cluster index. e , In-vivo effects of Serping1 overexpression on tumor volume. Left panel: B16-mhgp100 cells with Serping1 and vector-only overexpression were inoculated subcutaneously into mice treated by immune checkpoint blockade. Right panel: The tumor sizes on day 28, the day before the first tumor reached an endpoint (tumor volume ≥ 2000 mm or length ≥ 2 cm). Box plots are shown as in . Group values were compared through the two-sided Wilcoxon rank-sum test. f , Serping1 overexpression in tumors extended survival. B16-mhgp100 cells express immunogenic antigen hgp100, while the B16F10 cell line is less immunogenic. On the Y-axis, the fraction of mice with endpoint-free survival is plotted against days since tumor inoculation (X-axis). The two-sided log-rank test compared group survival differences.

Article Snippet: Spatial transcriptomics (ST) data paired with H&E images also makes it possible to implement a Transcriptomics-to-Image module, which returns image features associated with the high expression level of a query gene name (e.g., SERPING1 ) ( ).

Techniques: Gene Expression, Expressing, Activation Assay, Labeling, In Vivo, Over Expression, Plasmid Preparation